Switchover control techniques for dual-sensor camera system

ABSTRACT

Techniques are described for automatically selecting between multiple image capture subsystems with overlapping fields of view but different optical properties. A selection may be made by estimating a plurality of operational characteristics of an image capture event, and, based on those estimates, selecting a primary image capture subsystem for the image capture event. For example, in a device such as a cellphone comprising two capture subsystems, each subsystem including a lens system and sensor system where each subsystem has a different fixed optical zoom parameter, a subsystem can be chosen based on a combination of desired zoom value, estimated focus distance, and estimated scene brightness.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.15/617,918, filed Jun. 8, 2017, now allowed, which claims benefit under35 U.S.C. § 119(e) of U.S. Provisional Application Ser. No. 62/349,020,filed Jun. 12, 2016, which is incorporated herein by reference in itsentirety.

BACKGROUND

The present disclosure is directed to control systems for multi-cameraelectronic devices.

There are a wide variety of camera designs that find application inmodern consumer electronic devices. For example, compact cameras andsingle-lens reflex cameras tend to be fairly complex camera systems thatprovide control over a relatively wide range of operational parameters,such as focus distance, zoom, exposure time, iris control, shutter speedand the like. Other camera systems, particularly those that integrate acamera system into a multifunction consumer electronics device such as asmartphone, tablet computer or notebook computer, tend to be muchsimpler. They may provide a degree of control, for example, over focusdistance and/or exposure time but their designs may not permit variationof other parameters such as optical zoom or iris aperture. The level ofcomplexity of an individual camera system often dictates a costassociated with that camera system.

Traditionally, multifunction consumer electronics devices employedeither a single camera system or multiple camera systems looking inopposite directions to capture image data. The cameras of such deviceswere very simple, with limited focus capability and often no controlover optical zoom or other operational parameters. Recently, someconsumer devices have integrated multiple camera systems that haveoverlapping fields of view, each with different, fixed opticalproperties. For example, one camera system may have a relatively widefield of view and another camera system may have a narrower field ofview. In such cameras, users must expressly select which of the camerasystems is to be used for image capture. Such selections are cumbersomefor users and introduces delay into image capture operations, which cancause image capture opportunities to be lost if an imaging subjectchanges.

The inventors propose techniques to remedy these disadvantages.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a device according to an embodiment of thepresent disclosure.

FIG. 2 illustrates a first method of selecting between candidate imagecapture subsystems according to an embodiment of the present disclosure.

FIG. 3 illustrates a second method of selecting between candidate imagecapture subsystems according to an embodiment of the present disclosure.

FIG. 4 illustrates a third method of selecting between candidate imagecapture subsystems according to an embodiment of the present disclosure.

FIG. 5 illustrates a fourth method of selecting between candidate imagecapture subsystems according to an embodiment of the present disclosure.

FIG. 6 illustrates a fifth method of selecting between candidate imagecapture subsystems according to an embodiment of the present disclosure.

FIG. 7 is a system diagram of camera device according to an embodimentof the present disclosure.

DETAILED DESCRIPTION

Embodiments of the disclosure provide techniques for automaticallyselecting between multiple image capture subsystems with overlappingfields of view but different optical properties. A selection may be madeby estimating a plurality of operational characteristics of an imagecapture event, and, based on those estimates, selecting a primary imagecapture subsystem for the image capture event. For example, in a devicesuch as a cellphone comprising two capture subsystems, each subsystemincluding a lens system and sensor system where each subsystem has adifferent fixed optical zoom parameter, a subsystem can be chosen basedon a combination of desired zoom value, estimated focus distance, andestimated scene brightness. In the example, when the desired zoom value,estimated focus distance, and estimated scene brightness are all abovethresholds, the subsystem with higher optical zoom parameter value maybe chosen, while if any values are below its respective threshold, thesubsystem with a wider field of view (e.g. lower zoom parameter value)may be chosen as the primary camera system.

FIG. 1 is a block diagram of a device 100 according to an embodiment ofthe present disclosure. The device 100 may include a plurality of imagecapture subsystems 110, 120, a capture control system 130, an imageprocessing system 140, a storage system 150, and a user input system160. Each image capture subsystem 110, 120 may capture image datarepresenting image information of a local environment. The image capturesubsystems 110, 120 may have different optical characteristics owing todifferences in lens systems 112, 122 and/or sensor systems 114, 124therein. The capture control system 130 may estimate which of the imagecapture subsystems 110, 120 should be utilized as a primary camera tocapture image data during a capture event. Typically, the capturecontrol system 130 will alter operational parameters of the imagecapture subsystems based on the subsystems' status as a primary cameraor, alternatively, a secondary camera. The processing system 140 mayapply processing operations to the captured image data, which may bestored in the storage system 150. Operators of the device 100 mayinitiate a capture event by entering commands to the device 100 via theuser input system 160.

As indicated, each image capture subsystem 110, 120 may have a lenssystems 112, 122 and sensor system 114, 124. The lens systems 112, 122may include respective lenses that are driven by actuators (not shown)for focus control; such actuators may be voice coil actuators,microelectro-mechanical (commonly, “MEMS”) actuators or the like. Thelens systems 112, 122 optionally may include other camera elements suchas an iris (also not shown), which may control an amount of light thatis incident on the sensor system 114, 124.

The capture subsystems 110 and 120 may be oriented substantially in thesame direction and may have overlapping fields of view. For example,capture systems 110 and 120 both may be mounted on a housing H of thecapture device 100 in proximity to each other. Center axes of the lenssystems 112 and 122 may be aligned to each other, for example, asparallel or very close to parallel to each other, such that the capturesubsystems 110 and 120 would image data of a common scene if they wereenabled simultaneously.

In some embodiments, the capture control system 130 may controloperation of the lens systems 112, 114 by, for example, changing thefocus distance of the lens systems 112, 114. If the lens system is socapable, the capture control system 130 also may control zoom,magnification or field of view of the lens systems 112, 114, and/orcontrol an aperture diameter of light passed by an iris of the lenssystems 112, 114. It is unnecessary, however, that all of theseparameters of the lens systems 112, 114 be controlled by the capturecontrol system 130. And, of course, owing to the differences among thelens system 112 and the other lens system 114, the capture controlsystem 130 may be able to control some operational parameters in one ofthe lens systems (say, system 112) while those same operationalparameters would be fixed in the other lens system (say, system 122).Sensor system 114 may convert the light received by the sensor of sensorsystem 114 into a digital image comprising pixel data for a twodimensional raster array of pixels. Control system 130 may controlvarious aspects of the digitization process, such as controlling thesensitivity or gain of the sensor system 114.

Each sensor system 114, 124 may include a solid-state image sensor, forexample a focal plane array, a charge coupled device (commonly, “CCD”)or a CMOS image sensor. The image sensors 114, 124 each may have aplurality of pixel circuits defined therein that convert incident lightinto an electrical signal representing intensity of the incident light.Oftentimes, individual pixel circuits are coupled to optical filtersthat permit the pixel to detect a single component of incident light,for example, red, blue or green light. The sensor system 114, 124 mayinclude interpolators that generate multi-component image data at eachpixel location.

The image capture subsystems 110, 120 may have different characteristicsfrom each other. For example, both subsystems 110, 120 may have a fixedoptical zoom, but camera subsystem 110 may have a higher zoom level (andsmaller field of view) than the camera subsystem 120. Alternatively,both subsystems 110, 120 may have variable zoom, with camera subsystem110 having a higher maximum zoom than subsystem 120 and camera subsystem120 having a lower minimum zoom than subsystem 110. The camerasubsystems may also differ based on other characteristics, such as thequality of lenses in the lens systems, or the amount of power used bythe different capture subsystems. The capture control system 130 maydetermine settings of the sensor systems 114, 124 to define frameresolution and frame rates. For example, one of the capture subsystemsoutputs higher frame resolution and higher frame rate, while othersoutput lower resolution and rate to reduce memory and power consumption.When the functional range of a parameter value differs between thecamera systems, the functional ranges may but need not overlap eachother.

The capture control system 130, as its name implies, may controloperation of the image capture subsystems 110, 120 during captureevents. The capture control system 130 may alter operational parametersof the image capture subsystems 110, 120 during capture events, based ontheir status as primary cameras or secondary cameras. The capturecontrol system 130 may determine settings of the lens systems 112, 122to define the focus distance, zoom, magnification or field of viewand/or iris aperture diameter parameters discussed above. The capturecontrol system 130, therefore, may perform auto-focus operations byestimating degree of blur in output of the subsystems' image sensors114, 124 and correcting focus distance of the subsystems' lens systems112, 122. The capture control system 130 may determine settings of thesensor systems 114, 124 to define frame resolution and frame rates. Thecapture control system 130 also may perform auto-exposure control toregulate brightness of image content output by the sensor systems 114,124 as needed under changing brightness conditions.

As discussed, operational parameters of the image capture subsystems110, 120 may be altered based on their status either as a primary cameraor a secondary camera. In one embodiment, when an image capturesubsystem (say, subsystem 110) is selected as a primary camera, thatimage capture subsystem 110 may be enabled during the capture event andthe other image capture subsystem 120 may be disabled. Disablingnon-primary image capture subsystems 120 may conserve power in a device100. Alternatively, multiple image capture subsystems 110, 120 may beenabled during an image capture event but operational parameters of theimage capture subsystems 110, 120 may be tailored according to theirdesignations, either as primary cameras or secondary cameras. Forexample, a primary camera may be operated at a higher frame rate or ahigher resolution than a secondary camera.

The processing system 140 may be a processor or other device that formsan image file or video file as appropriate from output of the image dataoutput by the image capture subsystems 110, 120. For example, for stillimages, the processing system 140 may form files according to, forexample, one of the PNG (Portable Network Graphics), JPEG (JointPhotographic Experts Group), and/or EXIF (Exchangeable Image FileFormat) families of file formats. Similarly, for videos, the processingsystem 140 may form files according to, for example, the M4V, WindowsMedia Video or MPEG-4 families of file formats. The processing system140 may include one of more codecs 145 to perform compression operationsrequired by these file formats. Formation of such files may involveconversion of image data from native formats output by the image capturesubsystems 110, 120 to a format that is appropriate for the selectedfile format. The capture control system 130 and the processing system140 may enable partial operation for one of image capture subsystems,such as collecting auto-exposure/auto-focus/auto-white-balancestatistics, while disabling other functionality, such as the imageprocessing pipeline, to reduce memory and power consumption.

In an embodiment, the processing system 140 also may perform imageand/or video composition operations to merge outputs from the imagecapture subsystems 110, 120 into a common representation. For example,the processing system 140 may perform image stitching to merge a highresolution image obtained from a primary camera with a relatively narrowfield of view with a lower resolution image obtained from a secondarycamera with a wider field of view. The processing system also mayperform temporal interpolation to merge content from a primary cameraoperating at a relatively high frame rate but having a relatively narrowfield of view with a secondary camera operating at a lower frame ratewith a wider field of view. Thus, the processing system's output mayrepresent a merger of outputs from the image capture subsystems 110,120.

The storage system 150 may include one or more storage device to storethe image and/or video files created by the device 100. The storagesystem 150 may include electrical, magnetic and or optical storagedevices (not shown) to store these files.

The user input system 160 may include one or more user-actuated devicesto govern capture events. For example, the user input system 160 mayinclude a touch screen or mechanical button through which a userindicates that image capture is to commence or indicates when videocapture is to commence and when it is to end. The user input system 160also may include inputs, again perhaps a touch screen control orbuttons, that govern other aspects of image capture such as capture mode(video vs. still image, panoramic images vs. square images, etc.), zoomcontrol, flash control and the like.

During operation, the device 100 may capture digital still images ormotion video images by one of the capture subsystems 110, 120 asdetermined by the capture control system 130. Typically, at least one ofthe capture subsystems (say, subsystem 110) will be enabled as anoperator orients the device 100 for image capture. The capture subsystem110 may receive light from the scene being imaged. Lens system 112 maydirect received light to the sensor system 114, which may create imagedata therefrom. The image data from one image capture subsystem may berendered on a display (not shown) of the device 100 to preview imagedata that will be captured by the device. In other embodiment, the imagedata from two image capture subsystems may be combined to provideimproved experience. The operator may enter commands to the device viathe input system 160 to compose the image to suit the operator's needsand, when the image is properly composed, to cause the device 100 tocapture an image. The capture control system 130 may select which of theimage capture subsystems 110, 120 to perform image capture.

Control system 130 may choose which image capture subsystem 110, 120 touse for a capture a particular image, and control system 130 may alsocontrol any capture parameters that can be varied over a functionalrange, such as zoom or focus on the chosen lens system. The selection ofwhich capture subsystem is to be used may be overridden by input fromuser control 160 in which case the user control 160 may govern. In theabsence of an express selection of which image capture subsystem 110,120 is to be used, the capture controller 130 may select an imagecapture subsystem 110, 120 from a user's preferred image captureparameter values. For example, user control 160 may provide a desiredzoom level or a desired field of view, may specify a desired focusdistance, or may specify a desired scene brightness. For example, a usercontrol may specify a part of or point in a scene for which to optimizean image capture operation, and the focus distance to portion of a sceneor the brightness of that part of the scene can be estimated andoptimized for. Alternatively, the capture controller 130 may select animage capture subsystem 110, 120 for a capture event based on input fromother sensors 170 on the device 100, for example, a device orientationsensor.

The components illustrated in FIG. 1 represent components of an imagecapture system of the device 100. In practice, it is likely that adevice will include other components (not illustrated) that are used forother purposes. For example, in FIG. 1, the device 100 is shown as asmartphone, which may include processing systems and communicationsystems to permit exchange of data, including the image data captured bythe image capture system, with other devices via communication and/orcomputer networks such as cellular networks or short distance wirelessnetworks (e.g., WiFi, Bluetooth, etc.). Such other components are notillustrated in FIG. 1, for convenience.

FIG. 2 illustrates a first method 200 of selecting between candidateimage capture subsystems according to an embodiment of the presentdisclosure. Method 200 considers zoom and at least one other operatingcondition to select a camera or capture subsystem. If the zoom value isgreater than the threshold, then the method 200 may consider other imagecapture parameters, such as scene brightness or focus distance as itsselects an image capture subsystem should be primary.

The method 200 may find application in a system where the first cameraand the second camera differ based on operating parameters. For example,method 200 may be beneficial where the cameras differ based on theirfield of view, such as where the first camera system is a fixed zoom“wide field of view” camera and the second camera system is a fixed zoom“telephoto” camera. Under certain low light conditions, a digitallyzoomed image captured by the wide field of view camera may be betterquality an image having the same zoom value but captured by thetelephoto camera. Also, a wide angle camera may be able to focus oncertain distances that are outside of the focus range of the telephotocamera, and thus it may be desirable to select the wide camera asprimary to achieve the desired focus, even if the image may require moredigital zoom.

The method 200 may begin upon determination of a zoom value to beapplied during image capture (box 202). The method 200 may compare thezoom value to a threshold (box 204). If the zoom value is less than thethreshold, then the method 200 may select the wide field of view cameraas the primary camera (box 206). If the zoom value is greater than thethreshold (box 204), then the method 200 may evaluate an operatingcondition (or capture parameter) of a scene that would be captured (box208). The operating condition may be tested against a threshold (box210). If the threshold is not satisfied, the first camera may beselected as primary (box 206). If by contrast the respective thresholdis satisfied, is the second camera may be selected as primary camera(box 212).

The principles of the present disclosure may be expanded to includeseveral different operating conditions whose state may be estimated andcompared to respective thresholds. The thresholds may distinguishoperating conditions for which the first camera is preferred for use asthe primary camera from operating conditions for which the second camerais preferred for use as the primary camera. The selection of operatingconditions and their respective thresholds may be tailored to thedesigns of the camera subsystems that are at work in their devices afterconsideration of settings for which the various camera systemsout-perform the other camera systems.

FIG. 2 illustrates a choice between two camera capture subsystems,namely the first camera and the second camera. The first camera may be,for example, a wide field of view (FOV) camera (low zoom value), and thesecond camera may be a telephoto camera (higher zoom value). However,method 200 may be used for selecting between any two cameras, forexample where camera capture parameters vary by focus distance or lightsensitivity instead of, or in addition to, differing optical zoomparameters.

A scene brightness may be one of the at least one operating conditionsevaluated in box 208. The operations involving determination of zoomvalues (box 202) and estimation of scene brightness (box 208) may beperformed by activating one of the image capture subsystems and usingits output prior to image capture. For example, user zoom values andfocus distance may be derived as users compose images using output fromone of the image capture subsystems. Indeed, the method 200 may switchbetween the image capture subsystems dynamically as users alter suchfactors during image composition. For example, a user may enter adesired zoom value directly, such as by actuating buttons that governzoom values, moving a slider in a graphical user interface or entering apinch command via a touch screen devices. Alternately, a user couldspecify a numeric angle of a desired field of view. Similarly, focusdistance may be set during user composition as users cause the camera tofocus on a desired subject. Again, focus distance may be estimated basedon an auto-focus actuator current or may be read from a Hall positionsensor which measures lens placement directly.

FIG. 2 illustrates a variety of threshold values against which zoomvalues and other operating conditions are to be compared. Duringapplication, device designers may select threshold values that aretailored to the designs of the camera subsystems that are at work intheir devices after consideration of settings for which the variouscamera systems out-perform the other camera systems. For example, thezoom value threshold (box 204) may be defined after consideration ofoptical capabilities of the lens systems 112, 122 that are used in theirdevices. Similarly, a brightness threshold (box 210) may be definedafter consideration of noise sensitivities of sensor systems 114, 124(FIG. 1) being used in their devices. Thus, these threshold values maybe tailored to individual application needs.

In some applications, any of the thresholds may vary depending on one ormore factors, such as the desired zoom value, or which camera iscurrently the primary camera. As an example, the system may also includesome hysteresis, such as having a higher brightness threshold forswitching from a wide field of view camera as primary to a telephotocamera as primary than when switching from a telephoto camera as primaryto a wide field of view camera as primary. With a hysteresis such asthis may prevent constant switching of the primary camera when the scenebrightness is hovering around the scene brightness threshold.

FIG. 3 illustrates a second method of selecting between candidate imagecapture subsystems according to an embodiment of the present disclosure.Method 300 considers zoom and focus distance to select a camera orcapture subsystem. If the zoom value is greater than the threshold, thenthe method 300 may consider the focus distance as its selects an imagecapture subsystem should be primary. Again, the method 300 may findapplication in a system where the first camera and the second cameradiffer based on operating parameters. For example, they may differ basedon their field of view, where the first camera system is a “wide fieldof view” camera and the second camera system is a “telephoto” camera.

The method 300 may begin upon determination of a zoom value to beapplied during image capture (box 302). The method 300 may compare thezoom value to a threshold (box 304). If the zoom value is less than thethreshold, then the method may select a camera having wider field ofview as a primary camera for image capture (box 306).

If the zoom value is greater than the threshold, then the method 300 mayconsider other image capture parameters such as focus distance as itsselects an image capture subsystem to use. If the zoom value is greaterthan a threshold (box 304), the method 300 may estimate stability ofauto-focus operations (box 308) and may compare an auto-focus stabilityvalue to a third threshold (box 310). If the auto-focus stability valuedoes not exceed the third threshold, the method 300 may retrieve aprevious focus distance estimate used by the device (box 312).Alternatively, if the auto-focus stability value exceeds the thirdthreshold, the method 300 may generate an updated focus distanceestimate (box 314). Thereafter, the method 300 may determine whether thefocus distance is greater than a fourth threshold (box 316). If so, thenimage capture may occur using the image capture subsystem that has thetelephoto camera (box 318). If not, then image capture may occur usingthe image capture subsystem that has the wide field of view (box 306).

FIG. 3 illustrates a variety of threshold values against which zoomvalues, auto-focus stability and focus distance are to be compared.During application, device designers may select threshold values thatare tailored to the designs of the camera subsystems that are at work intheir devices after consideration of settings for which the variouscamera systems out-perform the other camera systems. For example, thezoom value thresholds (box 304), auto-focus stability thresholds (box310) and focus distance thresholds (box 316) may be defined afterconsideration of optical capabilities of the lens systems 112, 122 thatare used in their devices. Thus, these threshold values may be tailoredto individual application needs.

The operations involving determination of zoom values (box 302) andestimation of estimating focus distance (box 314) may be performed byactivating one of the image capture subsystems and using its outputprior to image capture. For example, user zoom values and focus distancemay be derived as users compose images using output from one of theimage capture subsystems. In some embodiments, the focus distance may beestimated based on an auto-focus actuator current or may be read from aHall position sensor which measures lens placement directly. Indeed, themethod 300 may switch between the image capture subsystems dynamicallyas users alter such factors during image composition. For example, auser may enter a desired zoom value directly, such as by actuatingbuttons that govern zoom values, moving a slider in a graphical userinterface or entering a pinch command via a touch screen devices.Alternately a user could specify a numeric angle of a desired field ofview. Similarly, focus distance may be set during user composition asusers cause the camera to focus on a desired subject.

Zoom values may be estimated from optical zoom factors, digital zoomfactors or a combination of both. Optical zoom factors may reflectoperational state of a variable focal length lens system in an imagecapture system that is so equipped. Digital zoom factors may reflectstate of processing operations to increase size of images captured by animage sensor beyond the optical capabilities of the image capturesystem. Digital zoom typically is accomplished by cropping an image inan area of interest and interpolating content within the cropped imageto meet pixel dimensions of a desired image size; such digital zoomoperations may be performed within an image capture subsystem 110, 120(FIG. 1) or alternatively by a processing system 140.

In some embodiments, the method 300 may apply different zoom valuethresholds (box 304) based on the amount of digital zoom and the amountof optical zoom that is active. For example, the threshold may beconfigured to select a first camera subsystem as a primary camera when adigital value as at a predetermined value, rather than to switch to asecond camera with a longer focal length and for which a lower amount ofdigital zoom would be applied.

FIG. 4 illustrates a third method of selecting between candidate imagecapture subsystems according to an embodiment of the present disclosure.Method 400 considers zoom and scene brightness to select a camera orcapture subsystem. If the zoom value is greater than the threshold, thenthe method 400 may consider the scene brightness as its selects an imagecapture subsystem should be primary. Again, the method 400 may findapplication in a system where the first camera and the second cameradiffer based on operating parameters. For example, they may differ basedon their field of view, where the first camera system is a “wide fieldof view” camera and the second camera system is a “telephoto” camera.

The method 400 may begin upon determination of a zoom value to beapplied during image capture (box 402). The method 400 may compare thezoom value to a threshold (box 404). If the zoom value is less than thethreshold, then the method 400 may select the wide field of view cameraas the primary camera (box 406).

If the zoom value is greater than the threshold, then the method 400 mayconsider other image capture parameters such as scene brightness as itsselects an image capture subsystem to use. If the zoom value is greaterthan the threshold (box 404), then the method 400 may estimatebrightness of a scene that would be captured (box 408) and compare thebrightness value to another threshold (box 410). If scene brightness islower than the second threshold, then the method 400 may select the widefield of view camera as the primary camera (box 406). If scenebrightness is greater than the second threshold, then the method 400 mayselect the telephoto camera as the primary camera (box 412).

The operations involving determination of zoom values (box 402) andestimation of scene brightness (box 408) may be performed by activatingone of the image capture subsystems and using its output prior to imagecapture. For example, user zoom values and focus distance may be derivedas users compose images using output from one of the image capturesubsystems. Indeed, the method 400 may switch between the image capturesubsystems dynamically as users alter such factors during imagecomposition. For example, a user may enter a desired zoom valuedirectly, such as by actuating buttons that govern zoom values, moving aslider in a graphical user interface or entering a pinch command via atouch screen devices. Alternately, a user could specify a numeric angleof a desired field of view. Similarly, focus distance may be set duringuser composition as users cause the camera to focus on a desiredsubject. Again, focus distance may be estimated based on an auto-focusactuator current or may be read from a Hall position sensor whichmeasures lens placement directly.

FIG. 4 illustrates a variety of threshold values against which zoomvalues and scene brightness are to be compared. During application,device designers may select threshold values that are tailored to thedesigns of the camera subsystems that are at work in their devices afterconsideration of settings for which the various camera systemsout-perform the other camera systems. For example, the zoom valuethreshold (box 404) may be defined after consideration of opticalcapabilities of the lens systems 112, 122 that are used in theirdevices. Similarly, a brightness threshold (box 410) may be definedafter consideration of noise sensitivities of sensor systems 114, 124(FIG. 1) being used in their devices. Thus, these threshold values maybe tailored to individual application needs.

FIG. 5 illustrates a fourth method 500 of selecting between candidateimage capture subsystems according to an embodiment of the presentdisclosure. Method 500 considers zoom, scene brightness, and focusdistance to select a camera or capture subsystem. If the zoom value isgreater than the threshold, then the method 500 may consider the scenebrightness as its selects an image capture subsystem. Again, the method500 may find application in a system where the first camera and thesecond camera differ based on operating parameters. For example, theymay differ based on their field of view, where the first camera systemis a “wide field of view” camera and the second camera system is a“telephoto” camera.

The method 500 may make three determinations of operational parameters,involving zoom value (box 502), scene brightness (box 504) and focusdistance (box 506), respectively. The method 500 may compare each ofthese estimated parameters to respective thresholds, shown in boxes 508,510 and 512, respectively. Based on these comparisons, the method 500may increase weight assignments to either the first camera (box 514) orthe second camera (box 516). When weight adjustments have been madebased on each of the three estimates, the method 500 may select thecamera having the highest overall weight as the primary camera (box518).

As illustrated in FIG. 5, when an estimated zoom value exceeds acorresponding threshold, the method 500 may increase a weight assignedto the second (perhaps telephoto) camera (box 516). In the contraryscenario, when the estimated zoom value does not exceed the threshold,the method 500 may increase a weight assigned to the first (perhaps widefield of view) camera (box 514).

Similarly, when an estimated scene brightness value exceeds acorresponding threshold, the method 500 may increase a weight assignedto the second (perhaps telephoto) camera (box 516). In the contraryscenario, when the estimated scene brightness value does not exceed thethreshold, the method 500 may increase a weight assigned to the first(perhaps wide field of view) camera (box 514).

Moreover, when an estimated focus distance value exceeds a correspondingthreshold, the method 500 may increase a weight assigned to the second(perhaps telephoto) camera (box 516). In the contrary scenario, when theestimated focus distance does not exceed the threshold, the method 500may increase a weight assigned to the first (perhaps wide field of view)camera (box 514).

As with the other embodiments, FIG. 5 illustrates a variety of thresholdvalues against which various operational parameters of the device are tobe compared. Again, device designers may select threshold values thatare tailored to the designs of the camera subsystems that are at work intheir devices. For example, a brightness threshold (box 510) may bedefined after consideration of noise sensitivities of sensor systems114, 124 (FIG. 1) being used in their devices. Zoom value thresholds(box 508) and focus distance thresholds (box 512) may be defined afterconsideration of optical capabilities of the lens systems 112, 122 thatare used in their devices. Thus, these threshold values may be tailoredto individual application needs.

FIG. 6 illustrates a fifth method of selecting between candidate imagecapture subsystems according to an embodiment of the present disclosure.Method 600 considers zoom, scene brightness, and focus distance toselect a camera or capture subsystem. In method 600, all these captureconditions must be greater than a threshold to select the second camera,otherwise the first camera is selected. Because of this, the firstcamera of method 600 may be considered a default camera. Again, themethod 600 may find application in a system where the first camera andthe second camera differ based on operating parameters. For example,they may differ based on their field of view, where the first camerasystem is a “wide field of view” camera and the second camera system isa “telephoto” camera.

After determining a zoom value (box 602) if the zoom value is less thana first threshold (box 604), the first camera is selected as primary(box 606). However, if the zoom value is greater than the threshold (box604), scene brightness may be estimated (box 608). If the estimatedscene brightness is less than a second threshold (box 610) then thefirst camera is selected as primary (box 606), otherwise focus distanceconsidered in boxes 614-622. Focus distance may be considered by firstestimating the stability of an auto focus system (box 614). If the autofocus system is stable, that is if the estimate is greater than a thirdthreshold (box 616), a current focus distance estimation is used (box620), otherwise a previous focus distance estimate is retrieved for use(box 618). The focus distance estimates used here may be averaged orlow-pass filtered estimates. The focus distance estimate determined ineither box 618 or 620 is tested against a fourth threshold (box 622). Ifthe focus distance estimate is greater than a threshold, the secondcamera may be selected as primary (box 612), otherwise, the first cameramay be selected as primary (box 606).

FIG. 7 is a system diagram of camera device 700, according to anembodiment of the present disclosure. The camera device 700 may includea processor 710, memory 720 and a plurality of image capture subsystems730.1, 730.2 all in communication with each other via a communicationbus. The device 700 may include a control system 740 that controls someoperational parameters of the image capture subsystems 730.1, 730.2. Thedevice 700 also may include other sensors 750, and user input/outputdevices (I/O) 760 that communicate with the processor 710 and/or memory730 by the communication bus.

The memory 720 may store instructions for execution by the processor 710that define an operating system 770 and one or more applications 780 ofthe device 700. The applications 780 may include a camera captureapplication 780.1, a photo view application 780.2, and optionally codecapplication 780.3, among others. Applications 780 and operating system770 be written in any programming language, such as, for example, C,Java, Objective-C, C+, Python, Visual Basic, Perl, or any otherprogramming language capable of producing instructions that are capableof execution on the processor 710.

The camera capture application 780.1, the operating system 770 and thecontrol system 740 may cooperate to control operation of the imagecapture subsystems 730.1, 730.2. For example, the camera captureapplication 780.1 may control timing of image capture events andparameters of image capture that are defined by user input. The controlsystem 740 may perform auto-focus and/or auto-exposure control of theimage capture subsystems 730.1, 730.2. With respect to selection betweenthe image capture subsystems 730.1, 730.2, the principles of the presentdisclosure may place such controls within either the camera captureapplication 780.1 or the control system 740 as may be convenient.

The control system 740 may be implemented separately from processor 710and may include dedicate hardware logic circuits. Control system 740 mayfurther include its own local memory for storage of, for example theimage capture parameters provided to it by the camera captureapplication.

The user I/O 760 components may include buttons or a touchscreencontroller to accept user input. They may include display devices torender image data during image composition or when reviewing capturedimage data.

The processor 710 can include, for example, dedicated hardware asdefined herein, a computing device as defined herein, a processor, amicroprocessor, a programmable logic array (PLA), a programmable arraylogic (PAL), a generic array logic (GAL), a complex programmable logicdevice (CPLD), an application-specific integrated circuit (ASIC), afield-programmable gate array (FPGA), or any other programmable logicdevice (PLD) configurable to execute operating system 770 andapplications 780 to facilitate capturing of images.

Memory 720 may be configured to store both programs and data. Asindicated, the memory 720 may store instructions for the operatingsystem 770 and applications 780 in machine readable form. The memory 720also may store data associated with applications 780. For example, thememory 720 may store image data captured by the camera captureapplication 780.1, device information, user information, and the like.The memory 720 may include computer readable storage media, for exampletangible or fixed storage of data, or communication media for transientinterpretation of code-containing signals. Computer readable storagemedia, as used herein, refers to physical or tangible storage (asopposed to signals) and includes without limitation volatile andnon-volatile, removable and non-removable storage media implemented inany method or technology for the tangible storage of information such ascomputer-readable instructions, data structures, program modules, orother data. In one or more aspects, the actions and/or events of amethod, algorithm, or module may reside as one or any combination or setof codes and/or instructions on a memory 720 or other machine readablemedium, which may be incorporated into a computer program product.

Other sensors 750 may include, for example, a device orientation sensorthat produces an estimate of the orientation of camera devices 700relative to the direction of gravity. Other sensors 750 may provide theinput from other sensors 170 of FIG. 1. User I/O 760 may include atouch-screen display, and may be capable displaying a captured image andmay be capable of receiving input from a user indicating when to capturea picture. User I/O 760 may provide user control 160 input of FIG. 1.Capture subsystem 730.1 may be subsystem 110 of FIG. 1, while capturesubsystem 730.2 may be subsystem 120.

The foregoing discussion has described operation of the foregoingembodiments in the context of a computer device such as a digitalcamera. Commonly, these cameras are provided as electronic devices suchas personal computers, notebook computers, mobile computing platformssuch as smartphones and tablet computers, dedicated gaming systems,portable media players, computer servers, and the like. As described,they may execute programs that are stored in memory of those devices andbe executed by processors within them. Alternatively, they can beembodied in dedicated hardware components such as application specificintegrated circuits, field programmable gate arrays and/or digitalsignal processors. And, of course, these components may be provided ashybrid systems that distribute functionality across dedicated hardwarecomponents and programmed general purpose processors, as desired.

Several embodiments of the disclosure are specifically illustratedand/or described herein. However, it will be appreciated thatmodifications and variations of the disclosure are covered by the aboveteachings and within the purview of the appended claims withoutdeparting from the spirit and intended scope of the disclosure.

We claim:
 1. A method of selecting between a plurality of image capturesubsystems during an image capture event, the image capture subsystemshaving different characteristics and overlapping fields of viewcomprising: comparing a focus distance value of a lens system to a firstthreshold; evaluating at least one other operating condition; comparingthe at least one other operating condition to respective additionalthresholds; selecting one of the plurality of image capture subsystemsas a primary camera based on the comparison to the first threshold andthe comparison to the respective additional thresholds.
 2. The method ofclaim 1, wherein one of the first operating condition and the at leastone other operating condition includes a scene brightness value of theimage capture event.
 3. The method of claim 1, wherein one of the firstoperating condition and the at least one other operating conditionincludes a zoom value of the image capture event.
 4. The method of claim1, wherein one of the first operating condition and the at least oneother operating condition includes a focus distance value of the imagecapture event.
 5. The method of claim 1, wherein the first operatingcondition and the at least one other operating condition include atleast a scene brightness value and a focus distance value of the imagecapture event.
 6. The method of claim 1, wherein the plurality of imagecapture subsystems includes a first subsystem having a first field ofview and a second subsystem having a second field of view, wider thanthe first field of view.
 7. The method of claim 1, further comprisingenabling the image capture subsystem selected as the primary camera forthe image capture event and disabling another image capture subsystem.8. The method of claim 1, further comprising operating the image capturesubsystem selected as the primary camera during the image capture eventat a higher frame rate than another image capture subsystem.
 9. Themethod of claim 1, wherein the evaluating comprises: enabling a selectedimage capture subsystem during the evaluating, evaluating the at leastone operating condition from output of the selected image capturesubsystem.
 10. The method of claim 9, wherein a different image capturesubsystem is enabled during the evaluating than during the image captureevent.
 11. The method of claim 9, wherein a same image capture subsystemis enabled during the evaluating and during the image capture event. 12.The method of claim 1 wherein the focus distance value of the lenssystem is specified by a user control.
 13. The method of claim 1,wherein the focus distance value of the lens system is estimated basedon Hall sensor measurement of the lens system.
 14. The method of claim1, wherein the focus distance value of the lens system is estimatedbased on auto-focus actuator current of the lens system.
 15. Anelectronic device, comprising: a first image capture subsystem having afirst image sensor, a second image capture subsystem having a secondimage sensor, wherein the first and second image capture subsystem havedifferent characteristics, a controller, responsive to a plurality ofestimated operational characteristics of an image capture eventincluding an auto-focus stability, to select either the first imagecapture subsystem or the second image capture subsystem as a primarycamera for the image capture event based a comparison of the estimatedoperational characteristics including the auto-focus stability torespective thresholds.
 16. The electronic device of claim 15, whereinthe estimated operational characteristics include at least a zoom valueand a scene brightness value.
 17. The electronic device of claim 15,wherein the estimated operational characteristics include at least azoom value and a focus distance value.
 18. The electronic device ofclaim 15, wherein the estimated operational characteristics include atleast a scene brightness value and a focus distance value.
 19. Theelectronic device of claim 15, further comprising a user input system toaccept input defining the onset of the image capture event.
 20. Theelectronic device of claim 15, further comprising a storage system tostore a file with content of the image capture event.
 21. Anon-transitory computer readable medium storing program instructionsthat, when executed by a processing device, cause the processing deviceto select between a plurality of image capture subsystems havingdifferent characteristics and overlapping fields of view for use duringan image capture event, by: comparing an auto-focus stability conditionto a first threshold; evaluating at least one other operating condition;comparing the at least one other operating condition to respectiveadditional thresholds; selecting one of the plurality of image capturesubsystems as a primary camera based on the comparison to the firstthreshold and the comparison to the respective additional thresholds.22. A method of selecting between a plurality of image capturesubsystems during an image capture event, the image capture subsystemshaving different characteristics and overlapping fields of viewcomprising: comparing an auto focus stability condition to a firstthreshold; evaluating at least one other operating condition; comparingthe at least one other operating condition to respective additionalthresholds; selecting one of the plurality of image capture subsystemsas a primary camera based on the comparison to the first threshold andthe comparison to the respective additional thresholds.
 23. A method ofselecting between a plurality of image capture subsystems during animage capture event, the image capture subsystems having differentcharacteristics and overlapping fields of view comprising: estimating abrightness value of a scene; comparing the brightness value to a firstthreshold; evaluating at least one other operating condition; comparingthe at least one other operating condition to respective additionalthresholds; selecting one of the plurality of image capture subsystemsas a primary camera based on the comparison to the first threshold andthe comparison to the respective additional thresholds.